UX research is at the cornerstone of UX design as it’s the best way to identify where there’s a problem and to uncover the design opportunities available to remedy them. Through various research methods, designers work to pinpoint the needs of their users and the best ways to meet those needs.
An important subcategory of UX research is quantitative user research. Quantitative user research is what most people imagine when they think of research data—it’s numerical or statistical information obtained through highly controlled testing environments. While its counterpart, qualitative data, gives researchers important subjective information, quantitative data provides crucial objective information that helps designers to evaluate their products and determine when a redesign might be necessary.
We’ve created this comprehensive guide to quantitative user research to help you understand what quantitative data is, how it’s used, and the different types of quantitative research methods employed by UX designers.
Here’s what we’ll cover:
- What is quantitative UX research?
- When to use quantitative UX research
- Quantitative UX research methods
- Utilizing mixed methods in UX research
- Key takeaways
Now, let’s get started!
1. What is quantitative UX research?
Quantitative user research is the process of collecting and analyzing objective, numerical data from various types of user testing.
Quantitative research utilizes large sample numbers to produce bias-free, measurable data about a user population. It answers the questions of “how many, how much, and how often?”
The main goal of quantitative user research is to indirectly measure the usability of a completed or final product. However, quantitative research is also used to compare a product to its competitors or calculate a company’s ROI.
2. When to use quantitative UX research
Quantitative user research is best conducted on a product that is already in existence or when you’re completing a final design. This means that quantitative research is utilized at the very beginning or very end of a design cycle. However, to time your quantitative research well, you’ll need to evaluate your research goals and understand the benefits that running quantitative user research will have on your designs.
Here’s a brief overview of some benefits of quantitative research, some potential downsides, and the situations when you should use quantitative methods in your UX research.
Benefits of quantitative research
- Easier to avoid human-bias as it’s difficult to lead participants to certain outcomes in a structured and highly controlled testing environment
- Allows you to assign numbers to the usability of your product, which may be more convincing to some investors and stakeholders
- Results in data that is easily presented in the form of graphs, charts, etc.
- Helps you understand if changes in your designs create different (statistically significant) outcomes
- Warrants less direct contact with users and easy to conduct remotely
Potential downsides to quantitative UX research
- Can only give you an idea of what is happening, but doesn’t answer the question of why certain trends or patterns are present
- Doesn’t result in anecdotal data or direct quotes from users, which can also be impactful with some investors and stakeholders
- In order to obtain any sort of statistical significance, you need to have a lot of respondents/participants (a large sample size), which may be difficult to organize
- Can be costly due to the need for a larger sample size
- Lets you know if a redesign is necessary but doesn’t clue you in on how the product should be redesigned
When to use quantitative UX research
- When comparing your product to its competitors
- To determine if a redesign is necessary or worth doing
- When you need to analyze and compare different designs and whether the changes observed are statistically significant
- If you need to evaluate company goals, key performance indicators, or when justifying support for and investment in your UX design team to upper management
3. Quantitative UX research methods
There are many types of tests and research methods that can provide meaningful quantitative data when analyzing your product. Here, we discuss a few of the most common ones: Analytics, heatmaps, funnel analysis, cohort analysis, and A/B testing.
Analytics
Photo credit: Google Blog
Analytics, and Google Analytics in particular, is one of the most valuable sources of quantitative user data. Through metrics like bounce rates, page views, conversion rates, and click-throughs, analytics helps give you a clear cut idea of the usability of a site and what users are doing when they interact with it. It can quickly give you numerical data that will suggest what parts of a site need work or if a complete redesign is warranted.
Mouse heatmaps
Photo credit: Clicktale
Similar to analytics, mouse heatmaps paint a pretty vivid picture (literally) of what is happening when a user interacts with a site. Mouse heatmaps help you visualize a user’s mouse movement and records when they hover, scroll, click, or pause when moving through a site. With radar-like color coding, mouse heatmaps show where your users’ attention is drawn and what areas can be leveraged for higher conversion rates.
Funnel analysis
Photo credit: Chartio
Funnel analysis is a method that helps you visualize the steps needed for your users to complete a task and analyze how successful they are at doing so at each step. The map typically takes on the shape of a funnel as you assess what percentage of users are making it all the way to the final step of the task. A funnel analysis helps you visualize where most users are dropping out of the journey and what steps in the funnel need attention in order to maximize conversion rates.
Cohort analysis
Photo credit: Google Support
Utilizing a cohort analysis helps you look at user engagement over time. The activity of your older users is often masked by the high rates of new users. So instead of lumping all your users into one category, a cohort analysis sorts them into related groups so you can accurately assess if engagement is improving over time or if it just appears that way due to growth. It will also show you where your users are dropping out and what areas need work in order to improve user retention.
A/B Testing
Photo credit: Seynd
A/B testing allows designers to compare two or more versions of a design on users to see which one is more effective. It’s best that the designs differ by only one or two key features so that you can accurately assess if different outcomes of the test are actually due to differences in specific features. While you can obtain some qualitative data from in-person A/B testing (ie. quotes, facial expressions), A/B testing gives you a percentage of how many users prefer each design, giving you a distinct idea of which version is more effective.
4. Utilizing mixed methods in UX research
Quantitative user research can offer invaluable data when analyzing the usability of a product. However, when used on its own, it’s not enough to portray a complete picture of how successful a product is at fulfilling the user’s needs. This is where mixed methods research comes in.
Mixed methods research simply refers to using both quantitative and qualitative user research to investigate the efficiency of a product.
As we’ve already said, quantitative data gives you objective, numerical data that answers all your “what, how many, and how much” questions. Qualitative data, on the other hand, provides crucial user feedback and subjective data to answer why certain trends are occurring, and how you might address any problems or pain points.
When used as a pair, quantitative and qualitative data give you a more thorough and detailed view of how users are reacting to a product, what areas have flaws or frictions, and how they might be remedied.
5. Key takeaways
Quantitative data is a crucial part of UX research that gives designers minimally-biased and objective numerical or statistical data about the efficiency of their products. Through various testing methods, UX researchers and designers employ quantitative user research to evaluate the usability of a product or determine if a redesign is necessary. However, quantitative UX research is best used alongside its qualitative UX research counterpart. When used in conjunction, designers can easily identify trends in user behavior, understand why they might be happening, and work towards resolving them.
To learn more about UX research, check out these articles: